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THE PRISM TABLEAU MODEL FOR SCHUBERT POLYNOMIALS 


ANNA WEIGANDT AND ALEXANDER YONG 


Abstract. The Schubert polynomials lift the Schur basis of symmetric polynomials into 
a basis for Z[xi, X 2 , ■ ■ ■]■ We suggesf the prism tableau model for these polynomials. A novel 
aspect of this alternative to earlier results is that it directly invokes semistandard tableaux; 
it does so as part of a colored tableau amalgam. In the Grassmarmian case, a prism tableau 
with colors ignored is a semistandard Young tableau. Our arguments are developed from 
the Grobner geometry of mafrix Schubert varieties. 
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1. Introduction 


1.1. Overview. A. Lascoux-M.-P. Schiitzenberger llLaSh82an recursively defined an inte¬ 
gral basis of Pol = Z[a:i, 0:2, ...] given by the Schubert polynomials : w G S'oo}- If wq is 
the longest length permutation in the symmetric group Sn then := ■ ■ ■ Xn-i- 

Otherwise, w ^ wq and there exists i such that w{i) < w{i + 1). Now one sets ©^ = 
where dif := (since the polynomial operators d, form a representation of Sn, this 

definition is self-consistent.) It is true that under the standard inclusion l : Sn ^ Sn+i, 
Thus one can refer to ©^ for each w G S^o = Un>i 

Textbook understanding of the ring Sym of symmetric polynomials centers around the 
basis of Schur polynomials and its successful companion, the theory of Young tableaux. 
Since Schur polynomials are instances of Schubert polynomials, the latter basis naturally 
lifts the Schur basis into Pol. Yet, it is also true that Schubert polynomials have noimega- 
tive integer coefficients. Consequently, one has a natural problem: 


Is there a combinatorial model tor Schubert polynomials that is analogous 
to the semistandard tableau model for Schur polynomials? 

Indeed, multiple solutions have been discovered over the years, e.g., l|Ko90L lBiJoSt93l, 
IIBeBi931, IIFoSt94ll , IIFoKi96i , iFoGrReSh97| , llMi98l, IIBeSo98[|BeSo02l, IIBuKrTaYo04l and 
IICoTal3l (see also iLaSh85l ). In turn, the solutions | BiJoSt93 |BeBi93ilFoSt94[|FoKi961 have 
been the foundation for a vast literature at the confluence of combinatorics, representation 
theory and combinatorial algebraic geometry. 

We wish to put forward another solution - a novel aspect of which is that it directly 
invokes semistandard tableaux. Both the statement and proof of our alternative model 
build upon ideas about the Grobner geometry of matrix Schubert varieties X^. We use the 
Grobner degeneration of X^j and the interpretation of ©„, as mutidegrees of X^, iKnMiOSl . 
Actually, a major purpose of loc. cit. is to establish the geometric naturality of the combi¬ 
natorics of [BiJoSt93| |BeBi93i |FoKi961 . Our point of departure is stimulated by later work 
of A. Knutson on Frobenius splitting liKn09i Theorem 6 and Section 7.2]. 


1.2. The main result. We recall some permutation combinatorics found in, e.g., in UMaOll . 

The diagram of w is D{w) = {{i,j) ■ 1 < b J < n,w{i) > j and w~^{j) > i} C n x n. Let 
Sss{w) C D{w) be the essential set of w: the southeast-most boxes of each coimected 
component of w. The rank function is rZi^j) = < i ■ u!{t) < j}. 


Define w to be Grassmannian if it has at most one descent, i.e., at most one index k 
such that w{k) > w{k + 1). If in addition w~^ is Grassmaimian then w is biGrassmannian. 
For e = (L j) G £ss{w) let Re be the {i — r^Fj)) x (j “ f'w{iij)) rectangle with southwest 
corner at position (i, 1) of n x n. The shape of w is \{w) = lJee£ss(io) 







....J ^ X(w) = 





Rei ^€2 


R, 


63 


Figure 1. The diagram of w = 35142 (with color coded essential set 
{ci, 62,63}), the overlay of Rg^, Re 2 , Res, arid the shape X{w). 
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A prism tableau T for w fills X{w) with colored labels (one color for each e G £ss{w)) 
such that the labels of color e: 

(51) sit in a box of Re', 

(52) weakly decrease along rows from left to right; 

(53) strictly increase along columns from top to bottom; and 

(54) are flagged: a label is no bigger than the row of the box it sits in. 

Let di{w) be the number of distinct values (ignoring color) seen on the i-th antidiagonal 
(i.e., the one meeting (i, 1)), for i = 1, 2,..., n. We say T is minimal if di{w) = t'(w), 
where E{w) is the Coxeter length of w. 

Let ic be a label i of color c. Labels {£c, id, ^e} ^ the same antidiagonal form an unstable 
triple ii i < i' and replacing the ic with i'^ gives a prism tableau. See Example 11.31 Let 
Prism(w) be the set of minimal prism tableaux with no unstable triples. Finally, set 

...,Xn):= = ]^xf 

TEPrisin(ti;) i 

Theorem 1.1. 6„,(xi,..., Xn) = ..., Xn). 

Example 1.2 (Reduction to semistandard tableaux). Consider the Grassmaimian permu¬ 
tation w = 246135. Conflating prism tableaux with their weights. Theorem 11.11 asserts: 
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Forgetting colors gives the following expansion of the Schur polynomial: 
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In general, if w is Grassmaimian then X{w) is a (French) Young diagram. Moreover, each 
cell of T G Prism(t(;) uses only one number. (See LemmaHd]) Replacing each set in T by 
the common value gives a reverse semistandard tableau. Thus = sx{w) follows. □ 

Prism tableaux provide a means to understand the i?G-graphs of iBeBi93[|FoKi96i . We 
think of the ^£ss{w)-many semistandard tableaux of a prism tableau T as the "disper¬ 
sion" of the associated i?G-graph through T. See Sections 4.1 and 4.3. 

Minimality and the unstable triple condition bond the tableau of each color, which is 
one reason why we prefer not to think of a prism tableau as merely a 7 )^£^ss(r(;)-tuple: 

Example 1.3 (Unstable triples). Let w = 42513. Then jj^8ss{w) = 3. The minimal prism 
tableaux and their weights are: 
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xlx2X^ 

X\X2X\ 




The second and the fourth tableaux have an unstably paired label. In both tableaux, the 
pink 1 in the second antidiagonal is replaceable by a pink 2. So = x\x 2 x\ -f xfx^xs. □ 
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Table 1. Prisni(w) and ©^ for all w G S '4 
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1.3. Organization. In Section 2 we present the general geometric perspective behind the 
rule and its proof. In the case at hand, we need to study the Stanley-Reisner simplical 
complex associated to the Grobner limit of this is done in Section 3. In Section 4, we 
collect some additional results and remarks. 


2. Main idea of the model and its proof 


Let G = GL„ and B and the Borel subgroups of lower and upper triangular matrices 
in G. Identify the flag variety with the coset space B\G. Let T be the maximal torus in B. 
Suppose X C B\G is an arbitrary subvariety and tt ; G ^ B\G is the natural projection. 
Then 

X = 7r-i(X) C MaGxn 

carries a left B action and thus the action of T. Therefore, one can speak of the equivariant 
cohomology class 

[X]t e HjiMatnxn) = . . .,Xn\. 

Moreover, the polynomial [Xjj is a coset representative under the Borel presentation of 

where is the ideal generated by (non-constant) elementary symmetric polynomials. 
This is a key perspective of work of A. Knutson-E. Miller IIKnMiOSl when X is a Schubert 
variety. 

LetF C Mat„xn be an equidimensional, reduced union of coordinate subspaces. Given 
V C n X n, we represent V visually as a collection of -l-'s in the n x n grid. We say P is a 

plus diagram for Y, if 

Cv := {M e Matnxn : Mij = 0 it (i, j) eV} CY. 

Let Plus(F) be the set of all such plus diagrams. Let MinPlus(y) be the set of minimal 
plus diagrams, i.e., those V for which removing any -|- would not return an element of 
Plus(y). We refer to the union of plus diagrams as an overlay to emphasize whenever 
(i, j) is in V or V, the diagram for V UV also has a -P in position (i, j). 

Each V corresponds 1 : 1 to a face of the Stanley-Reisner complex Ay. Let A„xn be the 
power set of {(z, j) : 1 < i, j < n}. Then Ay C A„xn and for each V one has the face 

= {(hi) : 1 < hi < rzand (z, j) ^ V}. 

The faces of Ay are ordered by reverse containment of their plus diagrams. Thus, facets 
(maximal dimensional faces) of Ay coincide with elements of MinPlus(F). In addition, 
taking the overlay of P G Plus(y) and Q G Plus(Z) corresponds to intersecting faces in 
the Stanley-Reisner complex: 

^VUQ — XQ & Ay n A^. 


Through the interpretation of [yjy as a multidegree, one may express [yjy as a generating 
series over MinPlus(F). That is. 


( 2 . 1 ) 


\Y]t = ^ wt(P), where wt(P) = 

'PGMinPlus(V) i=l 


# of -l-'s in row i 


Eor details, the reader may consult UMiStOSl : see Chapter 1 and Chapter 8 (and its notes). 
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Example 2.1. Let V C Mat2x2 be the zero locus of 12:1,2/ i-e., the union of fwo coordinate 
hyperplanes { 2 : 1,1 = 0} U {^ 1,2 = 0}. Then 




G pius(y) 


(the first two are in MinPlus(y')). The complex Ay is the 2-dimensional ball depicted 
below. , X 



Suppose -< is any term order on C[Mat„xn] and X' := init^A. Since X is T-stable 
the same is true of X'; thus [A']t is defined. Grobner degeneration preserves the T- 
equivariant class, so [X]j = [X']j. Suppose X' is reduced, and hence a reduced union 
of coordinate subspaces. Since X was assumed to be irreducible, then X is irreducible. So 
by llKaSt95[ Theorem 1] the Stanley-Reisner complex Ax' of X' is equidimensional. Hence 
we may apply the discussion above using Y = A' to compute [X']j = [X]j. 

We are interested in understanding Ax' under certain hypotheses on A. Assume that 
we have a collection of varieties A, Ai,..., Xm YV = such that 

(2.2) A = AinAan-'-nAfc. 

Assume -< is a term order on C[H] that defines a Grobner degeneration of these varieties 
so that each Grobner limit 

(2.3) X' := init^A, A' := init^Aj (for z = 1, 2,..., /c) is reduced. 

Finally, suppose 

(2.4) A' = A( n A' n ■ • • n A{. 

Call {Aj} a -< -spectrum for A. 

To construcf a cheap example, pick any Grobner basis G = {gi, ..., gM} with square- 
free lead ferms fo define A. Now partition G = Gi U G 2 U ■ ■ ■ U and set X^ to be cut 
out by Gfc. On the other hand, a motivating example is A. Knutson BKn09[ Theorem 6]: 
given a term order -< (satisfying a hypothesis), there is a stratification of V into a poset of 
varieties (ordered by inclusion) with the additional feature that each stratum A admits a 
-<-spectrum using higher strata. 

How can a -<-spectrum be used to understand the combinatorics of [A']t? Here is a 
simple observation: 

Lemma 2.2. Let {Aj} be a -<-spectrum for X. Then 

(1) Plus(A') = (p, u ■ ■ ■ U Pfc : P, G Plus(A').} 

(11) MinPlus(A') C (Pi U ■ ■ ■ U Pfc : Pi G MinPlus(A').} 
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Proof. (I): Let V G Plus(X'). Then £70 C X' C X' for all i. Therefore V G Plus(X') and 
trivially V = V U .. .UV, proving " C". For the other containment, suppose Vi G Plus(X') 
for 1 < i < fcandletP = ViU- ■ -UVk- Then£-p = £p^n...n£-pj^ and hence £-p C C X'. 
So P G Plus(X') for each i, which implies V G Plus(X'). 

(II): Let V G MinPlus(X'). By (I), V G Plus(X') for each i. Then there exists Vi G 
MinPlus(X') so that Vi C V. Then P 3 Pi U ■ ■ ■ U Pa; G Plus(X') by (I). As P is minimal, 
this is an equality □ 

Our point is that in good cases, the plus diagrams of X' are "simpler" to understand 
than those of X. Lemma [2.21 II1 says that one can think of each P G MinPlus(X) as an 
overlay P = Pi U ■ ■ ■ U P^ of these simpler P*. Of course, this representation is not unique 
in general, so one can make a choice of representation for each P. The hope is to transfer 
understanding of the combinatorics of MinPlus(Xj) to the combinatorics of MinPlus(X). 

3. Proof of the Theorem 11.11 

We now carry out the ideas described in Section 2 in the case of Schubert varieties. 

3.1. Matrix Schubert varieties and Schubert polynomials. The flag variety B\G decom¬ 
poses into Schubert cells := B\Br(;B+ indexed by w G S'„. The Schubert variety is the 
Zariski-closure X^ := X^. The matrix Schubert variety is 

:= 7r-i(X„,) C Mat,,xn- 

Let Z = izij)i<ij<n be the generic n x n matrix. The Schubert determinantal ideal is 

Iw = + 1 minors of the the northwest i x j submatrix of Z) C C[Matnxn]- 

In iFu91l Lemma 3.10] it is proved that 1^ cuts out X„, scheme-theoretically. Moreover in 
loc. cit. it is shown that is generated by the smaller set of generators coming from those 

(i, j) G £:ss(w). 

By iKnMiOSi Theorem A], 

• • • 5 ^n) ^ . 

Moreover, let -<anti be any antidiagonal term order on C[Mat„xn]/ i-c., one that picks 
off the antidiagonal term of any minor of Z. In IKnMiOSi Theorem B] it is shown that 
MinPlus(X(y) are in a transparent bijection with the PC-graphs of |BeBi931 (cf. IFoKi961 ). 

For each e G £ss{w), there is a unique biGrassmaimian permutation Ue such that (e) = 
r„,(e) and Sss{ue) = {e} ILaSh96l . Let 

biGrass(t(;) ;= {Ue : e G )} = {mi, ..., Uk}. 

Call {X„^,..., the biGrassmannian -<anti-spectrum for X^. By |Kn09l Section 7.2], 
{X„J indeed gives a -<anti-spectrum for X^ over Q. This result can also be readily ob¬ 
tained (over Z) if one assumes the Grobner basis result iKnMiOSi Theorem B]. (It should 
be emphasized that one of the points of iKn09l Section 7.2] is to reprove said Grobner 
basis theorem more easily.) 
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Figure 2. The Stanley-Reisner complexes for X 2314 intersect to 

give the complex for X 2443 . These complexes are a multicone over the de¬ 
picted complex. 


Example 3.1. X = X 2413 has biGrassmaimian -< anti-spectrum {Xi = Xi 423 ,X 2 = X 2314 }. 
Here C[Mat„xn] = : 1 < j < 4] and one can check: 


lui 


^1,1 2 : 1,2 
2^2,1 ^2,2 


2^1,1 ^1,3 

^ 2,1 2 : 2,3 


2:1,2 2:1,3 

2 : 2,2 ^2,3 


^ U 2 (2:1,15 2:2,1) 5 ^ U \ 4 ” 


The -<anti-Gr 6 bner limits are defined by 

^'ui = (2:2,12:1,2, 2:2,12:1,3, ^2,2^1,3), I'u2 = (2:1,1, 2:2,1), I'y, = I'u^ + I'u: 


Since the prime decomposition of is 

= (^2,1, ^2,2) n (^2,1, ^1,3) n (^1,2, ^1,3), 

the facets of Ax( are labeled by: 






■ . . + . ■ 



> 

(3.1) MinPlus(X() = < 


+ + • • 

5 

+ ■ • ■ 

5 










> 


In Figure ^ these correspond to the indicated tetrahedra. 

Similarly, there is a single facet for X'^ associated to the prime ideal J„ 2 , labeled by: 






> 

(3.2) 

< 


+ ■ ■ ■ 








This facet corresponds to the remaining tetrahedron. 
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There are precisely two minimal overlays of the plus diagrams of (13.11) with the plus 
diagram of (13.21) : 



’ + ■ ■ ■' 


I 

+ 

+ 

1 _ 

> 

< 

+ + ■ ■ 

5 

+ ■ ■ ■ 




T' 


y 


cally, these label the facets of Ax', pictured as light blue triangles in Figure |2l 
Finally, applying the discussion of Section 2 (cf. (12.11) 1 we see fhaf 

= x\ + X 1 X 2 + xl, &U 2 = X 1 X 2 , and = xix\ + x\x 2 

(where the terms in each Schubert polynomial correspond respectively to the plus dia¬ 
grams listed above). □ 


Example 3.2 (Digression: diagonal term orders). Fix w = 2143 G 5*4 and let -<diag be any 
diagonal term order on C[Mat„xn], be., any order that picks the diagonal term of a minor 
as the lead term. One has that A 2143 = A 2134 fi A 1243 (reduced intersection). Now 

-^2134 = (^11) 1(243 = (^11^22^33)- 


However, 

-^2143 = (2:11,2:122:212:33) 7^ -h 1(243 = (2:11,2:112:222:33) = -f(i34. 

So {X 2134 , A 1243 } is not a -<diag-spectrum for X^. □ 


A permufation is vexillary if if is 2143-avoiding; see UMaOll Section 2.2.1] for defails. 
The following is not needed in the proof of Theorem ll.il 

Proposition 3.3. ..., is a -<i^^g-spectrum for if and only ifw is vexillary. 

Proof Assume w is vexillary. Then by ||KnMiYo09i Section 1.4], the essential minors de¬ 
fine a -<diag-Gr 6 bner basis for I^. The same is frue of since Ui is biGrassmaimian and 
therefore also vexillary. Since the (Grobner) essential minors of are the concatentation 
of the (Grobner) essential minors of the .'s, the spectrum claim follows. 

For the converse, assume w is not vexillary, but ..., Xuf\ is a -<diag-spectrum for 
Xw Again, we know the essential minors of form a -<diag-Grobner basis. By the spec¬ 
trum assumption, the concatenation of these fc-many Grobner basis is a -<diag-Grobner 
basis for /„,. However fhis concatenafed Grobner basis is the set of essential generators 
for This direcfly contradicts iKnMiYo09l Theorem 6.1]. □ 


3.2. Multi-plus diagrams. The technical core of our proof is fo analyze fhe combinatorics 
of overlays of plus diagrams for the biGrassmarmian -<aiiti-spectrum {X„^,..., X„^}. Let 

k 

Multi(r(;) = MinPlus(X(jJ 

i=l 

be the set of multi-plus diagrams for w: we represent (Pi,..., Vk) G Multi(w) as a place¬ 
ment of colored -l-'s in a single n x n grid, where (a, b) has a -l- of color Ui if (a, b) E Vi. 

By Lemma IZ2l IL fhere is a map 

supp : Mult i(u;) -> Plus(X(„) 
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given by {Vi,,Vk) ^ ViU.. .UVk- Call ViU. . .UVk the support of (Pi ,... ,Vk). Central 
to our study is 

Multi(P) ;= supp~^(P). 

Example 3.4. Let w = 42513. Then biGrass(t(;) = {41235, 23415, 14523}. Now, 


V = 


G MinPlus(X(„). 


One can check that 




■ ++ ++ ++ ■ ■ ■ 


■ ++ + ++ ■ ■ ■ 




+ • + ■ ■ 


++ • 4- ■ ■ 


Multi(P) = < 


++ ■ ... 


++ ■ ■ ■ • 







> 


3.3. Local moves on plus diagrams. A southwest move is the following local operation 
on a plus diagram: 

(3.3) 

The inverse operation is a northeast move: 

(3.4) 

Suppose £:ss(«) = {(«, j)}. Define Pbot(M) G MinPlus(X(,) as the ii-ry,{ij))x{j-ru{ij)) 
rectangle of +'s, with southwest corner in row i and column 1. The following is well- 
known, and is a consequence (by specialization) of the chute and ladder moves of ||BeBi93i 
Theorem 3.7]: 

Lemma 3.5. Let u E Snbe MGrassmannian. 

(I) MinPlus(X{) is connected and closed under the moves (13.3P and (13.4P . 

(II) Each V E MinPlus(X{) can he obtained from Vbot{u) using only the moves (13.4P . 

Define a partial order on MinPlus(X{) by taking the transitive closure of the covering 
relation V < V ii V is obtained from P by a northeast local move (I3.4|) . Let <' be the 
partial order on Multi(w) defined as the £^ss(w)-factor Cartesian product of <. That is 
(Pi,..., Pfc) <' (Qi,..., Qk) if and only if Vi < Qi for each i. Then <' induces a partial 
order on Multi(P) C Multi(i/;). 

Given (Pi,..., Vm) G Multi(w), a long move is a repeated application of (13.31) (respec¬ 
tively, (13.41) ) to a single -I- appearing in one of the P/s. Recall that a lattice is a partially 
ordered set in which every two elements x and y have a least upper bound (join) x\/ y 
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(join) and a unique greatest lower bound x /\y (meet). It is basic that a Cartesian product 
of lattices is a lattice. 

Theorem 3.6. Let w E Sn and V G MinPlus(X(y). 

(I) Multi(P) is connected by long moves. 

(II) Each (MinPlus(X(jJ, <) is a lattice. Consequently, (Multi(ty), <') is a lattice. 

(III) (Multi(P), <') is a sublattice of {Kniti{w), <'). 

Example 3.7. Letw = 5361724. Fix 


+ 


V = 


+ 

+ 


G MinPlus(X(^). 


Here biGrass(w;) = {5123467, 3451267, 1562347, 1345627, 1256734}. Figure H shows the 
Hasse diagram for Multi(P). The poset is a lattice, agreeing with Theorem B.61 111). □ 


We use the following ordering of the +'s of V-botiu). 


(3.5) 


+7 

+4 +8 

+2 + 5+9 

+ 1 +3 +6 +10 


In words, order the +'s along diagonals, from northwest to southeast, where +i is at the 
southwest comer of "Dbot- 

In view of Lemma |331 II), there is a bijection between the +'s of 'Dbot(M) and any V G 
MinPlus(W(,). Hence the ordering (13.51) induces an ordering +i, + 2 ,... of the +'s of V. 

The following two lemmas hold by Lemma 1331 and an easy induction. 


Lemma 3.8. If in G MinPlus(X{) the label +„ is weakly southwest of+b, then the same 

is true for all V G MinPlus(X{). 


For an antidiagonal D, let Dien be the antidiagonal adjacent to D and to its left. Similarly 
let Hright be the antidiagonal adjacent to D and to its right. 

Lemma 3.9. Fix V G MinPlus(X{). Fix an antidiagonal D. Fet +„ G H and suppose +b is in 
D, Hieft or Height so that +b is weakly southwest of +a. Then for any V G MinPlus(X{), +?, is 
weakly southwest of+a. 

Proposition 3.10. (I) Fet ube a biGrassmannian permutation and V,V' G MinPlus(X(,). 

Consider the following lists of indices: 

SAME = (a : +a appears in the same location in V and V'), 

SW = (a : +a in V' is strictly southwest of+a in P), 
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Figure 3. The subposet of Multi(5361724) with support V (cf. Example 13.71) . 


and 

NE = (a : +a in V' is strictly northeast of + a in V). 

Let A he the sequence contained by the concatenation SAME, SW, NE where SAME and SW are 
listed in increasing order whereas the elements o/NE are listed in decreasing order. 
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(3.6) 


Then there exists a sequence: 

V \= Vi ^ 7^2 ^ ^ ^ Vh I—>■ Vh+i •—>■••• I—>■ Vi+i := V' 

where 

(i) ^ = ^{u) = |A(m)| 

(ii) each Vi G MinPlus(X^); 

(iii) Vh ^ Vh+i is an application of a long move to +\^. 

(II) (MinPlus(X(j), <) is a lattice. 

Example 3.11. Let u = 1267345. The ordering (I3.5D is as follows (where for brevity we 
ignore the unused bottom three rows of the 7x7 ambient square): 


'Pbot(M) = 


+2 +4 +6 

+ 1 +3 +5 


LetP = 

+2 

+4 +6 • ■ ■ 

and V'= 


+4 

+6 • 

+5 • 



. +1 +3 

+5 • ■ • 


+2 
. +1 

+3 




Here SAME = (1,4), SW = (2), and NE = (6, 5, 3). Therefore A = (1,4, 2, 6, 5, 3). 

Removing trivial long moves for SAME, the sequence (13.61) consists of the following 
moves (where we have underlined +a^ for emphasis): 








+2 

+4 +6 ■ • ■ 

1— y 7^4 — 


+4 ^ • 



+5 • • ■ 

+2 

+5 • 


+ 1 +3 



. +1 

+3 • • • 



+6 • ■ 


• +6 ■ • 

+4 • ... 

t—)■ Pe — 

+4 • +5 ■ • 

+2 • • ^ • ■ • 

+2. 

. +1 +3.. 


-1 

+1 

+ 


^ Vr 


■ • • • +6 

• +4 • +5 

+2 • +3 • 

+1 • • • • 


In particular, V 4 . = T’#same+#sw+i is P A V. 


□ 


Proof of Proposition \3.10\ (I): We show that for each 1 < h < i one can give the desired long 
move. Clearly, we can use the trivial long move for 1 < L < #SAME. 

For f^SME < h < #SAME + we have A/, G SW. Suppose the +a^ in V' is in row r' 
and suppose the +a^ in V is in row r (where we have assumed r' > r in matrix notation). 
Let D be the antidiagonal that +A;, sits in (in either V and V'). 
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Claim 3.12. In Vh, there is no +b in rows 

(1) r,r + 1,... ,r' - 1 of Dmu 

(2) r + 1, r + 2,..., r' of D^ght; or 

(3) r + l,r + 2,... ,r' of D. 

Proof of Claim \3.12\ II Ah = b then by definition the unique +a^ = +b is in row r of D, 
and in particular not in (1), (2) or (3). If Ah < b, by the definition of the ordering on +'s 
combined with Lemma [3.91 +b is not weakly southwest of +a^. Thus we may assume 
Ah > b. Since Ah> b the position of +;, in V' will either be the same (if b G SAME or 6 G SW), 
or strictly northeast of its position in Vh (if b G NE). The position of Ta^^ in V is row r' of 
D, which is weakly southwest of the position of +6 in V'. However, by the assumption 
that +b is in (1), (2) or (3), we see that in Vh, +& is weakly southwest of +a^- Hence we 
obtain a contradiction of Lemma lT9l □ 

In view of Claim |3T2] we may apply the long move Vh Vh+i that moves the Ta^, in 
row r of Vh to row r', showing (iii). Since each long move is by definition a composition 
of southwest moves, (ii) holds by Lemma |331 1). 

Finally, for A/, G NE we have a long move for fhe same reasons {mutatis mutandis) as in 
our analysis above of A/, G SW. (Alternatively, let W' = 'P#same+#sw+i and W = V. Then 
by the above arguments there are southwest long moves coimecting TV to TV'. Then we 
can reverse these moves to give the desired northeast long moves from W to V.) 

Since the list SAME, SW, NE is of length I, (i) holds trivially 

(11): We will only construct V A V' E MinPlus(X(j) (the construction olVyV is entirely 
analogous). From (1) we have the sequence of long moves (13.6D thaf fransform V into V. 
Lef V = Vh, where h corresponds to the first index in NE, i.e., h = #SAME + + 1. 

IZ is obtained from V by applying a series of southwest long moves and IZ < V. Like¬ 
wise, V' is obtained from TZ entirely by northeast long moves, so TZ < V. Suppose we 
have some other <-lower bound S oIV and V. We may construct T, a <-lower bound 
for TZ and S, by the same argument above we have used to construct TZ from V and V. 

Thaf TZ = V A V is immediate from the following: 

Claim 3.13. TZ > T = S. 

Proof of Claim \3.13\ We will work with the sets: 

A = {a : -|-a in 7^ is strictly southwest of -|-a in V} 

A' = {a : +a in TZ is strictly southwest of -|-a in V'} 

B = {a +araS is strictly southwest of -1-^ in P} 

S' = {a : -Pa in iS is strictly southwest of -Pa in V'} 

(7 = {a : -Pa in T is strictly southwest of -Pa in TZ} 

C = {a +a in T is sfricfly southwest of -Pa in 5} 

Summarizing, we have: 
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Since T < TZ < V and T < S < V, 

(3.7) y4 U C = {a : +a in T is strictly southwest of +„ inV} = B U C'. 

Similarly, 

(3.8) 71' U C = S' U C. 

By the construction of IZ and T we have 

(3.9) c n C" = 0 
and 

(3.10) AnA = ^. 

Intersecting both sides of (13.71) by C gives: 

(AuC) nc" = (su C) nc' ^ (Anc") u (Cnc") = (sn c') u (C"n c'). 

By (13.91) we have A n C" = C", which in turn implies C C A. Likewise, (13.8|) implies 
C C A'. Therefore by (I3.10|) , C" = 0 holds. This shows TZ>T = S, as claimed. □ 

This completes the proof of the proposition. □ 

Lemma 3.14. Fix P, V G MinPlus(X(j) (where u is MGrassmannian), and let 

P =: Pi I—)■ P2 '“t ... I—)■ I—)■ P/i+i I—)■ ... I—)■ P^ I—)■ Pf+i := P' 

he the sequence (13.6P in Proposition 13.101 (II). Then for any 1 < < £ + 1, P/, C P u P'. In 

particular, ifFZ = PAP' in the lattice (MinPlus(X(j), <) (where u is MGrassmannian), then 
P C P u P'. 

Proof. The claim about Vh follows from the construction of the sequence (13.61) in Proposi¬ 
tion |3A0KII). The claim about FZ holds since in the proof of Proposition 13.101 11), we have 
shown TZ = Vh (where h = ^SAIVIE -P #SW -P 1 ). □ 

3.4. Proof of Theorem 13.61 (I): Fix w G S'„, P G MinPlus(X(„). Let 

Q = (Pi,P 2 ,...,Pfc), Q' = (P(,P',...,PO G Multi(P). 

By Proposition 13.101 we may coimect Pi to V[ by long moves, as in (13.61) : 

Pi = Pl,l '“t Pi ^2 • ■ -Vl^h '“t Pl,/i+l I—)■ • • • I—)■ Pl ,£+1 = P(. 

By Lemma l3.14l PiC P^ u P( C P. Hence 

SUpp(Pi,/„ P 2 , • • • , Pfc) C P. 

Since P G MinPlus(X(„), this containment is an equality. That is, each (Pph, P 2 , • • ■ Pfc) ^ 
iy[ulti(P). In the case h = I + 1, we reach (P(, P 2 , P 3 , • • •, Pfc) from (Pi, P 2 , P 3 , • • •, Pfc). 

Continuing in this maimer, one connects 

(P(,P 2 ,P 3 ,...,Pfc)tO (P(,P',P 3 ,...,P,), 

by long moves, that keep one in iy[ulti(P), until one reaches (P(, P 2 , ■ ■ ■, Pfc)- 

(II) : This is Proposition 13.lOl II). 

(III) : Let Q, Q! G Multi(P), as in (I). Since by definition Multi(P) is a subposet of the 
lattice iy[ulti(r(;), it suffices to show Q A Q' G Multi(P). (The argument for the join is 


15 




















similar.) By Proposition 13.101 for each i, there is TZi = Vi A P'. In general, the meet in a 
Cartesian product of lattices is formed by taking the meet in each component. Therefore, 

QAQ' = (Pi, P2, ■ ■ ■, Pfc) G Multi(M;). 


By Lemma [3.141 TZi Vi U P( ^ V. Hence supp(Q A Q') C V. However, since V G 
MinPlus(P(„), we must have supp(Q A Q') = V, i.e., Q A Q' E Multi(P), as desired. □ 

The "MinPlus" hypothesis of Theorem 13. 6l is necessary, as we now demonstrate: 

Example 3.15 (Multi(P) for non minimal plus diagrams). Let w = 14253. Then we have 
biGrass(ty) = {14235, 12453}. Let 


V 


+ + 

• + 


GPlus(X;)\MinPlus(X;). 


Observe 







> 



+ +■■• 


+ +■■■ 


Multi(P) = < 




• + ■ ■ ■ 







> 


Multi(P) consists of two incomparable elements, so is in particular not a lattice. □ 

3.5. Conclusion of the proof of Theorem 11.11 By Lemma [2.21 IIL Multi(P) 7 ^ 0. In ad¬ 
dition, by Theorem 13.61 Multi(P) is a finite lattice and thus has a unique minimum Ai-p. 
Let 

Multi(t(;) := {A4-p : V E MinPlus(X{,)}. 

Hence, trivially, we have a bijection 

T : Multi(ty) —)■ MinPlus(X(y). 

Let AllPrisni(ty) denote the set oiall prism tableaux and MinPrism(r(;) the set of minimal 
prism tableaux for w. From the definitions, 

Prisni(t(;) C MinPrism(w) C AllPrism(w). 

Claim 3.16. There is a bijection 4) : AllPrism(t(;) — ?■ Multi(t(;). 

Proof. We associate each V E MinPlus (X{^) with a filling of Re. To do this, notice that by 
definition Re sits in n x n exactly as the -|-'s of V^otiue) do. Hence by Lemma 1331 11) there 
is a bijection between the -l-'s of V and the boxes of Re- 

Assign to each box of Re the colored label that is the row position of that boxes' associ¬ 
ated + mV. Then these labels of color e satisfy (SI) by definition. That they satisfy (S2) 
and (S3) follows from Lemma l331 Finally, (S4) holds by Lemma 1331 11). 

The map we have just described from V E MinPlus (X(j^) and the (S1)-(S4) fillings S 
of Re is clearly injective. That it is a surjection follows since the tableaux S are clearly 
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in weight-preserving bijection with the semistandard tableaux for 6^^ MMaOli Proposi¬ 
tion 2.6.8], which are known to be in bijection with P G MinPlus(X^ ), see, e.g., iKnMiYo09I 
Proposition 5.3] (and for an earlier reference, see iKoOOi ). 

Now, given T G AllPrism(t(;) we construct <P(T) := (Pi, ... ,Vk) G Multi(t(;) by ap¬ 
plying the above correspondence independently to each Rg. That this is a bijective map 
follows from the bijectivity on each component. □ 

Corollary 3.17. ^restricts to a bijection^ : MinPrism(t(;) supp~^(MinPlus(X(y)) C Multi(t(;) 

Proof. Since $ is a bijection, we are only required to show that 

im <l>|„inPrism(«;) = supp^^(MinPlus(X(„)). 

However, this holds, since a tableau T G AllPrism(w) is in MinPrism(w) if and only if 
supp(<l)(T)) has cardinality f'(w), i.e. if and only if supp(<l)(T)) G MinPlus(X(y). □ 

Claim 3.18. Let Q G Multi(t(;) and let T = <1>“^(Q). Then T has an unstable triple if and only 
if there exists a southwest long move Q' such that supp(Q) = supp(Q'). 


Proof of Claim \3.18] (^) Suppose T has an unstable triple {£c, (-d, d-'e} contained in an antidi¬ 
agonal D. Let T' be the tableau obtained by replacing with £(,. We must show Q differs 
from Q! ;= <I>(T') by a southwest long move such that supp(Q) = supp(Q'). If one could 
not conduct a southwest long move, there must have been some -P of color c in the region 
consisting of: 

(1) rows £ -P 1,..., f - 1 of Pieft. 

(2) rows £ -P 1, £ -P 2,..., f of P 

right • 

(3) rows £ -P 1, £ -P 2,..., P of P. 

Moving the -P of color c in row ^ to row P would cause it to appear southwest of +6/ 
contradicting Lemma |3]9] 

So now assume we have a southwest long move. It remains to check the support asser¬ 
tion. The labels Ic and ^db^\T each ensure there is a -P in row £ of P in supp(Q), while 
gives a -P to row of P in supp(Q). Similarly, i.d in T' corresponds to a plus in row £ of P 
in supp(Q'), while and gives each ensure there is a -P in row of P in supp(Q'). So 
replacing 4 in T with 4 in T' gives supp(Q) = supp(Q'). 

{^) Suppose we may apply a support preserving southwest long move to 

Q=(Pi,...,P„...,P,)^Q' = (Pi,...,P',...,P,). 

That is, there is an antidiagonal D C n x n such that Vc contains a -P in row £ of P that 
may be moved to row P > ihy a southwest long move. Since supp(Q) = supp(Q'), there 
must be colors d, e with the property that Vd has a -P in row £ of P and Ve has a -P is row 
P > i oi D. In T, this implies that there are labels {Ic, Id, Pff in P. Let T' be obtained from 
T by replacing 4 with 4- Then T' = $(2) G AllPrisni(t(;). So {45 45 4} is an unstable 
triple. □ 


Claim 3.19. $ (further) restricts to a bijection, $ ; Prism(w) —)■ Multi(w). 


Proof Since we know <I) is a bijection, we need only show that im <L|prism(«)) = Multi(t(;). 

If Mj, G Multi(t(;), then M-p is by definition the minimum in Multi(P). Let T := 
(this exists by Corollary I3.17|l . By Claim |3T8] (=^), if T has an unstable triple. 
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then there exists a southwest long move Mp ^ Q' so that supp(A^p) = supp(Q'). But 
then Q! <' M.p, a contradiction. Hence T G Prisni(t(;). Thus, im $|prisin(io) ^ Multi(ty). 

Suppose T G Prism(t(;) C MinPrism(t(;), and let Q := $(T) G supp“^(MinPlus(X^)). 
Suppose Q is not the minimum element in Multi(supp(Q)). Then there exists a southwest 
long move Q Q', so that supp(Q) = supp(Q'). TTren by Claim |3T8] («^), T must have 
had an unstable triple, contradicting T G Prism(t(;). Thus, Q = A^supp(Q) G Multi(w). This 
shows im <l>|prisin(io) ^ Multi(r(;), as required. □ 


By Claim l3T9l T o $ : Prisni(w) —)■ MinPlus(X^) is abijection. Now, 


wt(T) = JJ 


x. 


# of antidiagonals containing i 


and wt(P) = 


X- 


# of +'s in row i 


That wt(T) = wt((\l/ o <I>)(T)) is immediate from these definitions. Hence the theorem 
follows. □ 


4. Further discussion 


4.1. Comparisons to the literature. Ultimately, the evaluation of any model for Schubert 
polynomial rests on its success towards the Schubert problem, i.e., finding a generalized 
Littlewood-Richardson rule for Schubert polynomials. Due to the analogy with Sym, one 
hopes that a solution will not only provide merely a rule, but rather lead to an entire 
companion combinatorial theory. This would presumably enrich our understanding of 
Pol and its role in mathematics just as the Young tableau theory does for Sym. 


That the prism model manifestly uses Young tableaux is our impetus for ongoing in¬ 
vestigations that fundamental tableaux algorithms might admit prism-generalizations. 

The first rule for Schubert polynomials was conjectured by BKo90ll . This rule begins 
with the diagram of w and evolves other subsets of n x n by a simple move, the Schubert 
polynomial is a generating series over these subsets. A proof is presented in IIWi99l[Wi02i . 
Arguably, this rule is the most handy of all known rules, even though the set of Kohnert 
diagrams does not have a closed description. 


Probably the most well-known and utilized formula is given by |BiJoSt93 [, which ex¬ 
presses the Schubert polynomial in terms of reduced decompositions of w. This rule is 
made graphical by the i^C-graphs of ||BeBi93l (cf. IIFoKi96l ). One can obtain any RC- 
graph for w from any other by the chute and ladder moves of |BeBi931 . 


While neither of the above rules transparently reduces to the tableau rule for Schur 
polynomials, it is not too difficult to show in either case, that the objects involved do 
biject with semistandard tableaux, see IIKo90l and IKoOOi respectively. 


We are not aware of any published bijection between the Kohnert rule and any other 
model for Schubert polynomials. On the other hand, there is a map between the prism 
tableaux and i^U-graphs: the labels on the i-th antidiagonal indicate the row position of 
the -|-'s on the same antidiagonal in the associated i?U-graph. This map is clearly injective 
but we do not currently have a purely combinatorial proof that the map is well-defined. 
Tracing our proof of the main theorem, well-definedness comes from the Grobner basis 
theorem of IKnMiOSi . Moreover, in said proof, we treat each i?G-graph as a specific overlay 
of i?G-graphs for bigrassmaimian permutations. The latter i?G-graphs are in bijection 
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with semistandard tableaux of rectangular shape. This is the reason for fhe "dispersion" 
remark of the introduction. 


The work of IIFoGrReSh97| gives a fableau rule for Schuberf polynomials of a differenf 
flavor. This rule freafs 6^ as a generating series for balanced fillings of the diagram of w. 
The reduction fo semisfandard fableaux for Grassmarmian w seems non-trivial. 


In ||BuKrTaYo04l , a formula is given for a Schubert polynomial as a normegative integer 
linear combination of sum of products of Schur functions in disjoint sets of variables (with 
nontrivial coefficients). This is also in some sense a tableau formula for &yj. In IILe041 
this result is rederived as a consequence of the crystal graph structure on i?G-graphs 
developed there. 


4.2. Details of the reduction to semistandard tableaux. We now explicate the reduction 
from prism fableaux fo ordinary semistandard tableaux, as indicated in the introduction. 

Proposition 4.1. Assume w E Sn is Grassmannian. 

(I) The shape \(w) is a Young diagram, in French notation. 

(II) Let T E MinPrism(t(;). All labels of a box ofT have the same number. 

(III) T does not have unstable triples, i.e., MinPrism(r(;) = Prism(t(;). 

Proof. (I): Since w is Grassmarmian, it has a unique descent, > w(k + l). Furthermore, 
all essential boxes of w lie in the fcth row, say in columns ai < ... < aj. The rectangle R{k,ai) 
starts at row r^(k, af + 1, which strictly increases as i increases, since essential boxes to 
the right in the diagram take on higher values for the rank function. Each rectangle is left 
justified by construction, and has width ai — r^ik, Oi). This value strictly increases with 
each i, since 

T (^^(kj ci,-|-i) r'yj(k.j ®i)) ^ 

as seen from the diagram of w. So the rectangles R{k,ai) overlap to form fhe shape of a 
partition. 

(II): Suppose nof. Let x be a "bad" box, i.e., one with ic and in x where I f I' (and 
thus c f d). We may assume x is the northeast-most bad box. Let D be the antidiagonal 
containing x. We may also assume that each box of D contains a label of color c. 

Case 1: (P > l)\ Then by (S2) and (S3) fhe labels of color cmD, fhaf are sfricfly northeasf 
of x, are all distincf and differenf fhan both I and P . For the same reason, all labels of 
color d in D sfricfly southwesf of x are disfincf and differenf fhan I and P . Hence fhe fofal 
number of distincf numbers in D exceeds Since | A(tc) | = I(w), we conclude T carmot 

be minimal, a contradiction. 

Case 2: (P < i): Again by (S2) and (S3), all labels of color c that are in D but strictly 
southwest of x are distincf and are also different than i and P. Hence if D is not overfull 
(i.e., has more disfincf numbers than boxes) must appear in a box y in H that is strictly 
northeast of x. By the definition of prism tableaux and \{w), either there exists: 

(1) a box z in the row of x and in the column of y fhaf confains labels rric and or 

(2) a box w in the column of x and in the row of y fhaf confains labels rric and mf 

We may assume the first case occurs (the argument for the other case is the same). In 
view of the E y combined with (S3), m > P . On the other hand, in view of the P^ E x 
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combined with (S2), m' < t'. Hence we see z is a bad box strictly east of x, a contradiction 
of the extremality of x. 

(Ill): Suppose T has an unsfable triple {^cAdA'e} antidiagonal D. Let T' be the 
tableaux obtained by replacing 4 with Then by definition, T' G Prism(t(;). By (II), 
^c must sit in a box containing no other labels. By the definition of X{w), fhis furfhermore 
implies every box of H in X{w) has a label of color c. (II) then implies the box containing 
fg must contain a label This contradicts (S2) and (S3) combined. □ 


4.3. Stable Schubert polynomials. The stable Schubert polynomial (also known as the 
Stanley symmetric polynomial) is the generating series defined by 

F^{xi,X 2 , ...) := lim 

m—>-oo 

where itwESn then 1™ x w is the permutation in Sm+n defined by 

(I*" X w){i) = i for 1 < i < m and (1™ x w){m + i) = m + w{i) for 1 < i < n. 

It is true that 

^2t ■ ■ ■ 1 ^mi 0, 0, . . .) ©im (xi , . . . , Xyjii 0, 0, . . .). 

Now, notice that A(l™ x w) and X{w) are the same shape, but the former is shiffed down 
m sfeps in the grid relative to A(w). Therefore if follows fhaf 

F^{xi,X2, ... ,a;m,0,0,...) = ^wt(T), 

T 

where the sum is over all unflagged (i.e., exclude (S4)) minimal prism tableaux of shape 
X{w) that use the labels 1, 2,..., m. In the limit, this argument implies the generating 
series F^(a;i, 0 : 2 ,...) is given by the same formula, excepf we allow all labels from H. 


4.4. An overlay interpretation of chute and ladder moves. In IIBeBi93i , chute moves 

were defined for pipe dreams. These moves are locally of the form 


(4.1) 


V 


. + + + ■•• + + 

+ + + + ■•• + + 




+ + + ■•• + + 

+ + + ■•• + + 



Q 


Suppose V G MinPlus(X(„), biGrass(t(;) = {ui, ... ,Uk} and V = Vi U ■ ■ ■ U Vk, where 
Vi G MinPlus(A(jJ. We now show: 

The chute move's "long jump" of a single + may be interpreted as a sequence of the 
northeast local moves (13.3P applied to the Vfs. 

Example 4.2. Lefty = 1432. Now, biGrass(t(;) = {ui = 1423, tt 2 = 1342}. Consider fhe 
following sequence of northeasf moves 



Lef the support of the first and third plus diagrams be V and Q, respectively. We have 
V, Q E MinPlus(X4). V and Q differ by a chufe move. At the level of fhe overlays, one 
sees this transition as an application of (13.3D to each blue + in the second row. □ 
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Example 14.21 indicates the general pattern. Let (i, j) be the position of the southwest + 
of P in (|4.H) and {i — 1, j') the position of the northeast + of Q in (I4.1|l . Without loss of 
generalify, we may assume each Vi,... ,Vt contains a + at (i, j) while all other Vh do not. 

Claim 4.3. Consider the interval of consecutive +'s in row i of Vh (I- < h < t) starting at the left 
with the + in position {i,j). One can apply the move (13.4P (in the right to left order) to each of 
these +'s to obtain V[,..., V[. 

Proof. It follows from Lemma 13.91 thaf the configurations given below do not appear in 
MinPlus(X^) whenever u is biGrassmarmian: 



■ + 


■ + 


+ + 

1 


+ + 


+ ■ 


+ ■ 

i 


Suppose there is an obstruction to one of fhe local moves. If musf appear in row i — 1. 
Due fo the + in position (i, j), such an obstruction necessarily forces one of the above 
configurations to appear, causing a contradiction. □ 

Claim 4.4. V^ C Qfor 1 < h <t and Vh ^ Qfor t + 1 < h < k. 

Proof. Lirst suppose 1 < h < t. Each move from Claim |43] fakes a + from position (i, a) 
with j < a < j' and replaces it with a + in position (z — 1, a + 1) G Q. Lurthermore, each 
V'f^ has no + in position (z, j). So C Q. If /z >t + l, then by assumption Vh has no + in 
position (z, j). So Vh C P\{(z, j)} C Q. □ 

Claim 4.5. Q = P^ U ■ ■ ■ U p; U Vt+i U ■ ■ ■ U P^. 

Proof. Let Q = V[C .. .V[C Vt+i ... U P^. Suppose Q f Q. By Claim |4^ each P' C Q, 
for 1 < z < f and Vi C Q for t + 1 < i < k. Then Q ^ Q E Plus(X^), confradicting the 
assumption that Q G MinPlus(X^). □ 

A similar discussion applies to the ladder moves. 

4.5. Future work. It is straightforward fo assign weights to prism tableau in order to give 
a formula for double Schuberf polynomials. 

A generalization fo Grothendieck polynomials requires a deeper confrol of fhe over¬ 
lay procedure. In investigating this, one is led to some results of possibly independenf 
inferesf. 

Specifically, for Theorem 11.11 we have used the fact that the facets of Ax^ are infersec- 
fions of facets of those associated to biGrass(t(;). One can make a similar conjecture for 
all interior faces w's complex. Each Ax^ is a ball or sphere IIKnMi04l Theorem 3.7]. Hence 
one can refer fo the interior faces of this complex. Let 

IntPlus(t(;) = {P : P G Plus(z(;) and Pp is an interior face of Ax^}. 

Conjecture 4.6. IntPlus(z(;) C {Vi U ■ ■ ■ U P^ : P* G IntPlus(Mj), for Ui G biGrass(zz;)}. 

Conjecture I4.6l has been exhaustively computer checked for all rz < 6. 

As part of an intended proof of Conjecture 14.61 one defines A-theorefic analogues of 
the chute and ladder moves of iBeBi93] : thaf is if P —)■ Q by a chufe move (respectively, 
ladder move) then P ^ P U Q is a A-chute (respectively, A-ladder move). Whereas not 
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all interior plus diagrams are connected by the original chute and ladder moves, it is true 

that they are connected once one allows the extended moves. 

The first author plans to address these and related issues elsewhere. 
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